Index(['Date', 'Time', 'ReqDomain->ProDomain', 'Requester->Provider',
       'Data_Shared', 'ReqClass', 'ProClass', 'AccessPermission', 'PIV',
       'DirectTrust', 'ResponseTime', 'Context', 'Location', 'PacketErrorRate',
       'FaultTolerance', 'QoS', 'Availability', 'Reliability', 'Security',
       'Privacy', 'AggregatedTrustValue(AfterProcessing)', 'Comments',
       'Current Hash', 'Previous Hash'],
      dtype='object')

<AxesSubplot:>

<AxesSubplot:>

<AxesSubplot:>

<AxesSubplot:>

<ipython-input-10-c96b47432e0e>:5: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  DataWithoutATV['SUM'][DataWithoutATV['SUM'].values>1] = 1

Date                     object
Time                     object
ReqDomain->ProDomain     object
Requester->Provider      object
Data_Shared              object
ReqClass                 object
ProClass                 object
AccessPermission         object
PIV                     float64
DirectTrust             float64
ResponseTime            float64
Context                 float64
Location                float64
PacketErrorRate         float64
FaultTolerance          float64
QoS                     float64
Availability            float64
Reliability             float64
Security                float64
Privacy                 float64
Comments                 object
Current Hash             object
Previous Hash            object
SUM                     float64
TrustStatus               int64
dtype: object

(9398, 12)

<seaborn.axisgrid.PairGrid at 0x7fe33256bb50>

DecisionTreeClassifier()

RandomForestClassifier(n_estimators=25)

<ipython-input-34-f29c187af440>:2: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
  plt.figure(dpi=600)

BaggingClassifier(base_estimator=DecisionTreeClassifier(), max_samples=0.5,
                  n_estimators=20)

/opt/anaconda3/envs/notebook/lib/python3.8/site-packages/sklearn/utils/validation.py:72: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  return f(**kwargs)

AdaBoostClassifier(base_estimator=DecisionTreeClassifier(), learning_rate=1,
                   n_estimators=10)

VotingClassifier(estimators=[('lr', LogisticRegression()),
                             ('dt', DecisionTreeClassifier()),
                             ('svm', SVC(probability=True))])

              precision    recall  f1-score   support

           0       0.90      0.96      0.93       190
           1       1.00      0.99      0.99      1690

    accuracy                           0.99      1880
   macro avg       0.95      0.98      0.96      1880
weighted avg       0.99      0.99      0.99      1880

	Date	Time	ReqDomain->ProDomain	Requester->Provider	Data_Shared	ReqClass	ProClass	AccessPermission	PIV	DirectTrust	...	FaultTolerance	QoS	Availability	Reliability	Security	Privacy	AggregatedTrustValue(AfterProcessing)	Comments	Current Hash	Previous Hash
0	6/11/2019	5:19:12	11->33	1008(Banking)->3007(AmbulanceServices)	56	CU	CU	Allowed	0.0	0.000000	...	0.000000	0.014901	0.043246	0.011930	0.000000	0.000000	0.117591	HighlyDistrusted	0	0
1	6/11/2019	5:19:13	11->22	1003(Entertainment)->2007(MotionDetection)	0	NC	CU	Allowed	0.0	0.000000	...	0.000000	0.000000	0.000000	0.091545	0.049339	0.000000	0.145219	HighlyDistrusted	y05q4isu33193yrcshwp04vzxlk6zwt65miaz36epdi7fe...	0
3	6/11/2019	5:19:13	22->11	2002(GPS)->1004(OrderStatus)	Delivering	CNU	NC	Allowed	0.0	0.007644	...	0.000000	0.005358	0.044182	0.673732	0.000000	0.000000	0.800000	HighlyTrusted	igplhb8lij60dfljkrfg1w50luw4zh7qedcx9du54bdr9r...	oxitw1nflfw6q1a01kbbwyi7jtp9hv2wxwnxgfwvco18iu...
4	6/11/2019	5:19:13	33->33	3005(V2VMessaging)->3001(InstantMessaging)	CallBack	CU	NC	Allowed	0.0	0.065993	...	0.000000	0.080646	0.000000	0.000000	0.000000	0.017155	0.227152	HighlyDistrusted	jysq6ur13u0trj03ywg9f9r83hzyteynakeeztagfub6q9...	igplhb8lij60dfljkrfg1w50luw4zh7qedcx9du54bdr9r...
5	6/11/2019	5:19:13	33->11	3008(DrugManagement)->1011(SmokeDetector)	Detected	CNU	CU	Allowed	0.0	0.000000	...	0.037583	0.031334	0.024597	0.087589	0.000000	0.000000	0.800000	HighlyTrusted	vr9g8ofs88nf3kdt0bqd6rb7854ifhqjt8rhq72y4ea352...	jysq6ur13u0trj03ywg9f9r83hzyteynakeeztagfub6q9...
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
14994	6/11/2019	5:20:00	11->11	1007(Uber)->1005(FoodDelivery)	Delivered	CNU	NC	Allowed	0.8	0.004950	...	0.000000	0.011569	0.065547	0.000000	0.000000	0.000000	0.800000	HighlyTrusted	qgrhxnlo3nrfrniiql4nxgmfs5xxmxczodobcy6xzq0ank...	ndgj43zr0inmkvsbnul862109bditvkc8j56r58byarrva...
14995	6/11/2019	5:20:00	22->11	2006(Temperature)->1002(Gaming)	NFS	CU	NC	Allowed	0.8	0.000491	...	0.000000	0.000000	0.048003	0.062911	0.066431	0.008323	0.800000	HighlyTrusted	lah9co89fmthzk2tagtwjk28t91ghpygeplf7111ewgpzs...	qgrhxnlo3nrfrniiql4nxgmfs5xxmxczodobcy6xzq0ank...
14998	6/11/2019	5:20:00	33->11	3001(InstantMessaging)->1005(FoodDelivery)	Delivering	NC	NC	Allowed	0.8	0.010078	...	0.000000	0.097540	0.156241	0.000000	0.000000	0.000000	0.800000	HighlyTrusted	gx7gyb84u74v3nxkwvbpivcy0wkxmz674dejm5kiyn31hk...	7pti0dca1bi0yxushyl23gzcqdwamgfeazijg4apvdvm2h...
14999	6/11/2019	5:20:00	33->11	3006(PatientMonitoring)->1011(SmokeDetector)	Detected	CU	CU	Allowed	0.8	0.000000	...	0.069537	0.042032	0.016714	0.071711	0.000000	0.000000	0.800000	HighlyTrusted	3l3oxafqvd49z684byq561yyo5vygddfx63zgg8nwphxvj...	gx7gyb84u74v3nxkwvbpivcy0wkxmz674dejm5kiyn31hk...
15000	6/11/2019	5:20:00	33->11	3010(BodyTemperature)->1011(SmokeDetector)	Detected	CU	CU	Allowed	0.8	0.000000	...	0.143384	0.028766	0.044643	0.090515	0.000000	0.000000	0.800000	HighlyTrusted	4y7046ype5hkfwirn17shet4bzbx2cd3h83erzfl3i94wk...	3l3oxafqvd49z684byq561yyo5vygddfx63zgg8nwphxvj...

	PIV	DirectTrust	ResponseTime	Context	Location	PacketErrorRate	FaultTolerance	QoS	Availability	Reliability	Security	Privacy	AggregatedTrustValue(AfterProcessing)
count	9398.000000	9398.000000	9398.000000	9398.000000	9398.000000	9398.000000	9398.000000	9398.000000	9398.000000	9398.000000	9398.000000	9398.000000	9398.000000
mean	0.670797	0.026401	0.103937	0.035511	0.043228	0.014511	0.025441	0.038966	0.053154	0.049047	0.019855	0.014945	0.724831
std	0.252213	0.039505	0.207362	0.117903	0.046756	0.027032	0.033686	0.035687	0.047946	0.119514	0.030521	0.027165	0.176684
min	0.000000	0.000000	0.000006	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.002445
25%	0.757588	0.000000	0.031723	0.000000	0.000000	0.000000	0.000000	0.000000	0.006863	0.000000	0.000000	0.000000	0.800000
50%	0.800000	0.004485	0.063292	0.000000	0.032303	0.000000	0.000000	0.033644	0.045754	0.011866	0.000000	0.000000	0.800000
75%	0.800000	0.044767	0.094320	0.019770	0.072953	0.016100	0.049330	0.067921	0.085276	0.059511	0.036638	0.018110	0.800000
max	0.800000	0.174981	1.747957	0.949954	0.249281	0.099986	0.149891	0.174849	0.174972	0.949089	0.099982	0.099757	0.800000

	Models	Accuracy	TestScores	TrainScores
0	DT	0.978191	0.978191	1.000000
1	BG	0.985106	0.985106	0.995877
2	RF	0.989894	0.989894	0.999867
3	ADB	0.976596	0.976596	1.000000
4	VT	0.985106	0.985106	0.990157

	Date	Time	ReqDomain->ProDomain	Requester->Provider	Data_Shared	ReqClass	ProClass	AccessPermission	PIV	DirectTrust	...	Reliability	Security	Privacy	SUM	TrustStatus	DT	BG	RF	ADB	VT
0	6/11/2019	5:19:12	11->33	1008(Banking)->3007(AmbulanceServices)	56	CU	CU	Allowed	0.000000	0.000000	...	0.011930	0.000000	0.000000	0.217591	0	1	1	1	1	1
16	6/11/2019	5:19:14	11->11	1002(Gaming)->1006(MobilePackages)	MessageBundles	NC	NC	Allowed	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.243511	0	1	1	1	1	1
22	6/11/2019	5:19:14	22->11	2004(SmartSprinkling)->1001(InternetAccess)	allow	CU	NC	Allowed	0.000000	0.057054	...	0.067028	0.091491	0.068350	0.542697	1	1	1	1	1	1
38	6/11/2019	5:19:15	11->11	1006(MobilePackages)->1003(Entertainment)	Youtube	NC	NC	Allowed	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.269270	0	1	1	1	1	1
43	6/11/2019	5:19:15	11->11	1010(SmartDoorbell)->1004(OrderStatus)	Delivering	CNU	NC	Allowed	0.000000	0.008216	...	0.388881	0.000000	0.000000	0.690454	1	1	1	1	1	1
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
1838	6/11/2019	5:20:41	33->11	3005(V2VMessaging)->1002(Gaming)	NFS	CU	NC	Allowed	0.800000	0.013985	...	0.072682	0.045639	0.005975	1.000000	1	1	1	1	1	1
1846	6/11/2019	5:20:41	33->22	3001(InstantMessaging)->2004(SmartSprinkling)	0	NC	CU	Allowed	0.306366	0.000000	...	0.000000	0.000000	0.000000	0.496785	0	1	1	1	1	1
1850	6/11/2019	5:20:41	22->11	2004(SmartSprinkling)->1004(OrderStatus)	Delivering	CU	NC	Allowed	0.800000	0.013160	...	0.918588	0.000000	0.000000	1.000000	1	1	1	1	1	1
1874	6/11/2019	5:20:42	11->22	1010(SmartDoorbell)->2004(SmartSprinkling)	0	CNU	CU	Allowed	0.056239	0.000000	...	0.000000	0.000000	0.000000	0.341394	0	1	1	1	1	1
1876	6/11/2019	5:20:42	33->22	3006(PatientMonitoring)->2004(SmartSprinkling)	0	CU	CU	Allowed	0.800000	0.000000	...	0.000000	0.000000	0.000000	0.981846	1	1	1	1	1	1

Importing neccessary modules¶

Reading and modefying data¶

Correlating Data¶

Normal¶

Pearson when 9300 observations are correlating¶

Kendal when 9300 observations are correlating¶

Spearman when 9300 observations are correlating¶

By looking above result spearman is better than other 3 methods¶

Now as seen above All the features have less correlation with target values¶

Data (max,min,avg)¶

Dropping AggregatedTrustValue from data and modifying outranged Rows sum to 1¶

Classifying data as Trustworthy or Untrustworthy¶

Classifying Trustworthy and Untrustworthy rows¶

Datatypes of varibles in modified data¶

Features we need for modelling¶

Splitting data and Training and Testing¶

Decision Tree¶

RandomForest Classifier¶

Bagging¶

Adaboost Classifer¶

Voting Classifier¶

Model Comparisons¶

End¶

SUM Represents AggregatedTrustValue¶